Computerized image system

ABSTRACT

This system and process for providing a rendered image of a virtual 3-D scene comprises an image generating system which stores therein data defining the 3-D scene. Responses to receiving control data such as viewing parameters, editing commands, or instructions to a software application working with the scene data, the visualization systems renders a pixel data image of the scene and transmits the pixel data over a data network, such as the Internet, to a user computer to be viewed on a display device, such as a monitor.

FIELD OF THE INVENTION

[0001] This invention relates to the field of electronic transmission ofcomputer generated images, and, more particularly, to systems andmethods by which computers interact over a computer data network totransfer data for displaying computer-rendered images for viewing by auser.

BACKGROUND OF THE INVENTION

[0002] Computer data networks, and particularly the Internet, allowusers to access software on separate computers located either nearby orat great distances from the user. These remotely-accessed softwareapplications sometimes involve display to the user of computer-renderedimages which represent views of virtual three-dimensional scenes.

[0003] A number of systems have been developed which provide for fairlyquick turnaround time for preparation and display of rendered images ofvirtual scenes. The rate of image rendering can even be fast enough thatimages can be rendered in sequence at a rate that can approximate orachieve real-time graphics interaction of the user with the remoteapplication.

[0004] The most common of these high-speed graphics rendering systemsare based on dedicated “graphics pipeline” hardware in the user'scomputer. In these graphics pipeline systems, the remote applicationserver transmits simple scene data over the network to the usercomputer. This scene data typically comprises data defining a group oftriangles, or “primitives”, that make up the objects of thethree-dimensional scene. At the user computer, the scene data is sent toa separate graphics pipeline circuit board of the user computer, such asOpen GL graphics H/W sold by Silicon Graphics or NVIDIA.

[0005] The graphics hardware renders a rasterized image representing aview of the virtual scene from the scene data by a series of discreteand independent computation steps. The circuit performs thesecomputation steps like an assembly line, or “pipeline”, or parallelcalculations in which several images are simultaneously proceedingthrough the circuitry in varying states of completion. This circuitprovides for rendering of serial images separated by close increments oftime, which provides a flow of images that is fast enough to approximatesequential frames of video.

[0006] Unfortunately, although images can be rendered fairly quickly andclose together by graphics pipeline hardware, the resulting images arevery coarse and unrealistic. For example, usually the image does notcontain any shadows, and the surfaces have an unrealistically uniformquality.

[0007] At the same time, increasing demand for more complex scenes isincreasing the amount of scene data needed for a 3-D scene, which placeseven greater demands on the limited power of pipeline graphics cards.Improvements in communications protocols have increased datatransmission rates, but the amount of scene data that is desired fortransfer is becoming prohibitively large, despite these increases. As anexample, the actual image pixel data for frames of the film “Star Wars:the Phantom Menace” each represented about 25 megabytes of data, whilethe scene data that defined the scenes ran as high as approximately 10to 15 gigabytes of data, i.e., hundreds of times larger.

SUMMARY OF THE INVENTION

[0008] It is therefore an object of the present invention to provide amethod and a system that provides for interactive rendering of imagesfor three-dimensional scenes over data networks, preferably withturnaround times that are suitable for real-time interactivity.

[0009] According to the methods and systems of the present invention, auser computer transmits control data over a computer data network, suchas the Internet, to an image generating computer system. The imagegenerating system automatically uses the control data to influence scenedata defining a three dimensional scene and/or optical parameters forviewing the defined scene.

[0010] Instead of transmitting the scene data to the user computer forrendering an image using the user's system, as has been done in thepipeline graphics systems of the prior art, the image generating systemrenders pixel data at a location remote from the user, which datarepresents a view of the three-dimensional scene based on the scene dataand the viewing parameters. The image generating system transmits thepixel data of the image over the data network to the user computer, andthe user computer then displays the pixel data to the user as a viewableimage.

[0011] The pixel data may be for a single image, transmitted as rawpixel data, or in a compressed form, such as in JPEG format. In thepreferred embodiment, however, images are rendered by the imagegenerating system in a continuing stream and transmitted for viewing asreal-time interactive video, preferably in MPEG format.

[0012] This system is particularly advantageous where the size of thescene data needed to render the pixel image exceeds the size of the datapackage needed to define the pixel image, and where the computationalpower required to render the pixel image within an acceptable turnaroundtime exceeds the computational capacity of the user system.

[0013] Preferably, a large number of users are connected with the imagegenerating system, and these users may be widely distributedgeographically. To improve real-time operation, the image generatingsystem preferably includes a discrete geometric system that maintainsthe scene data. The geometric system is connected by very high-speed,high volume data transmission lines to a number of separate,geographically distributed visualization systems. Each visualizationsystem communicates with a respective user computer over the datanetwork, and is preferably located close in the network to the usercomputer to reduce latency in the network communications links.

[0014] To further improve access of the visualization systems to thescene data in a widely distributed system, the geometric system may bemade up of a plurality of geographically distributed computer systemsconnected by high speed data links.

[0015] It is further an object of the invention to provide an imagegenerating system with improved rendering techniques and systems.

[0016] It is also an object of the invention to provide acomputer-accessible storage device or memory storing a databasecontaining the scene data organized in a way which facilitates thedetermination of what visual objects are encountered along a paththrough the scene or in the neighborhood of a location therein.

[0017] It is also an object of the invention to provide a method fordetermining the objects in a scene that lie in a path using theaforesaid stored database.

[0018] Other objects and advantages of the invention will becomeapparent from the specification herein.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019]FIG. 1 is a schematic view of the multi-user image generatingsystem of the invention.

[0020]FIG. 2A is a detailed schematic showing one of the visualizationsystems and its link to a user computer.

[0021]FIG. 2B is a schematic showing an alternate embodiment of an imagegenerating system in which single-frame rendering is provided at slowerturnaround times.

[0022]FIG. 3 is a perspective view diagram showing a three-dimensionalvirtual scene with visual objects therein.

[0023]FIG. 4 is a front elevational view of the scene of FIG. 3, towhich multiple uniform subdivision partitioning has been applied.

[0024]FIG. 5 is a diagram of a hash table used to store scene data.

[0025]FIG. 6 is a diagram of a scene divided into cells.

[0026]FIG. 7 is a diagram as in FIG. 6 with a further subdivision.

[0027]FIG. 8 is a diagram as in FIG. 7, showing multiple, virtualuniform subdivision (“MVUS”).

[0028]FIG. 9 is a diagram of the use of the hash table entry foraccessing a record of a complex visual object which is repeated invarious scenes.

[0029]FIG. 10 is a diagram of the use of the hash table entry toidentify the visual objects in a cell.

[0030]FIG. 11 is a diagram of a rotated nested sub-cell in a larger MVUSgrid.

[0031]FIG. 12A is a flowchart of the algorithm for tracing a ray throughthe space defined by a hash table as shown in FIG. 5.

[0032]FIG. 12B is a detail flowchart showing the traversal algorithmused in the flowchart of FIG. 12A.

[0033]FIG. 13 is a diagram of a virtual scene, illustrating the imagerendering process of the invention.

[0034]FIG. 14 is a diagram of the image generating system in whichgraphics pipeline circuitry is part of the image rendering system.

[0035]FIG. 15 shows a raster silhouette, the first step in the pipelinecalculation for determining illumination of a point from the lightsource in FIG. 13.

[0036]FIG. 16 shows a second step of the calculation, wherein anillumination texture is mapped on the light source silhouette of FIG.14.

[0037]FIG. 17 shows a further step, wherein the effect of transparentobjects is combined with the light source silhouette of FIG. 15.

[0038]FIG. 18 shows the light source of FIG. 16, with occluding opaqueintervening objects as viewed incorporated into the silhouette.

[0039]FIG. 19 illustrates the BRDF model of viewing of a reflected lightor a visual object.

[0040]FIG. 20 is a diagram of reflected light intensity as a function ofvarying viewpoint.

[0041]FIG. 21A shows the viewed intensity of reflected light as afunction of varying angles of incident light from the light source,mapped onto a sphere surrounding the point of reflection.

[0042]FIG. 21B shows the viewed intensity of reflected light as in FIG.21A, but mapped onto a hemi-cube around the point of reflection.

[0043]FIG. 22A shows the hemi-cube of FIG. 21B partially unfolded into aplane.

[0044]FIG. 22B shows the unfolded hemi-cube of FIG. 22A organized as atexture field for transmission to a graphics card.

[0045]FIG. 23 shows a result of a rendering process according to theinvention wherein the rasterized light pattern is prepared with theraster frame in the same orientation during all light sourcecalculations.

[0046]FIG. 24 shows a result of a rendering process according to theinvention wherein the rasterized light pattern is rotated randomlybefore the raster processing.

[0047]FIG. 25 shows the combination of the light source raster patternof FIG. 17 with the reflection texture of the illuminated point, toyield the light for the point rendered.

[0048]FIG. 26 shows a image rendered according to the invention from thescene of FIG. 13.

DETAILED DESCRIPTION

[0049] In the systems and methods here described, an interactive imagegenerating system communicates over a network with users to provide themwith images rendered for scene data defining a three dimensional virtual“scene.” The image generating system stores the scene data so as to makerendering more efficient, and the rendering process used is alsoimproved by incorporating pipeline-style steps in the software orhardware without sacrificing image quality.

[0050] a. The Interactive System As best seen in FIG. 1, a computerimage generating and displaying system according to the inventioncomprises a geometry control system generally indicated at 3 connectedwith a number of geographically distributed visualization systems 5.Each visualization system communicates over a link 15 via a datanetwork, such as the Internet, with at least one user computer 13.

[0051] The geometric control system 3 comprises a number of individualgeometry computer systems 7, shown in FIG. 1 to be three systems, butpotentially, and preferably much greater in number. The geometry controlsystem 3 maintains and processes scene data which defines one or morethree-dimensional scenes from which computer rendered images can beprepared and displayed to the various users 13.

[0052] The scene data may be distributed among the geometry computersystems 7 of the geometry control system 3 in a variety of ways.Preferably, each geometry system 7 has scene data for separate scenes,and each geometry computer 7 has access to every scene on the variousgeometry computer systems 7 via high-speed communications lines 9. Whenone of the geometry computer systems 7 alters the scene data, it advisesthe other geometry computer systems 7 of the changes to the scene databeing made over communications lines 9, and each geometry system 7 isthus kept up to date.

[0053] The high speed data links 9 are preferably dedicated, hard-wireddata links that transfer data at an extremely high rate, either onsingle channels or in parallel, as is well-known in the art. The datalinks connect all of the geometry systems 7, with each other, eitherdirectly or through other geometry systems 7. Alternatively, thegeometry system 7 can be in the same network location as a connectedvisualization computer 5.

[0054] The individual geometry computer systems 7 each communicate withthe visualization computer systems 5 using further high speed data links11. Data links 11 are preferably also dedicated hard-wired links whichallow for a very high rate of data transmission, preferably on the orderof at least about 10 megabits/sec, and more preferably above about agigabit/sec. The visualization systems 5 may be thousands of milesapart, either within a single country or across internationalboundaries, or may be clustered together, even in a single installation,where a group of users are close together. As a result of the distanceand number of network hops potentially present, latency even inhigh-speed communication may be an issue when the communication is via adata network or some other link with an inherent time delay, e.g., asynchronous satellite link. To reduce latency, the individual geometrycomputers 7 are therefore preferably distributed geographically andlocated on a data network to be close thereon to the associatedvisualization computer systems 5, i.e., within at most 2 or 3 networkhops of each other.

[0055] The visualization computers 5, in turn, are each connected with arespective user computer 13 by computer data network links 15. Thelinkage 15 may be through individual local area networks (LANs) but aremost preferably through the Internet or cellular phone networks. Thelinks 15 may also be telephone or cable TV links between the usercomputers 13 and the visualization computer 5. As a result, the links 15to the various user computers 13 may not be completely separate links,but may overlap with each other or share some communication pathways,such as where all of the links 15 are Internet communications.Similarly, a number of visualization systems 5 may be tied in to thesame local area network and communicate with user systems 13 on thenetwork.

[0056] The individual visualization computer systems 5 each receivesscene data from the geometry system 3 and produces an electronic imagetherefrom. The image represents a view of at least a portion of thescene defined by the scene data, prepared pursuant to data defining apoint of view that corresponds to the specific user, and viewingparameter data for rendering the scene. The electronic image ispreferably pixel data, but may be any other viewable image format,including analog images, such as cable TV signals, layered depth images,or any 2D or 3D image format.

[0057] The image which is produced by the visualization system 5 istransmitted over the data link 15 to the associated user computer 13. Asbest shown in FIG. 2A, the visualization system 5 includes acommunications connection or other hardware data linkage 17communicating via high-speed data link 11 to one of the geometrycomputers 7 of the geometry system 3. Visualization system 5 alsofunctions as a server, having communication connection 19 which linksthe visualization system 5 to a computer data network, generallyindicated at 21. The data network preferably is the Internet, a wirelessnetwork, (e.g., cellular), or a broadband network, e.g., a cable networkor DSL, but may also be a local area network (“LAN”) or other type ofcomputer data transmission network.

[0058] Each user computer 13 comprises the computer components familiarin the art, such as a computer CPU 23 and input devices for introducinginstructions into the computer CPU, such as keyboard 27 and mouse 29. Adisplay 31 is connected with the computer CPU 23 and displays imagestransmitted thereto in a form which is viewable by the human userthereof. The computer 13 may also be provided with a microphone inputand/or an audio output system.

[0059] The user computer 13 also includes a communication connection 25which is connected with the data network 21 for communicating with theassociated visualization system 5. The nature of the network and theconnection is preferably such that streaming serial images, e.g., VHS-or HDTV-quality video, can be transmitted to the user computer 13.

[0060] To decrease latency in the network link between the user and thevisualization computer, the visualization computer 5 is desirablylocated as close as feasible in network proximity to the user computer13. Although the system may advantageously be used with much largernetwork separation, a connection of less than 2 or 3 network hops ispreferred for a real-time interactive system with streaming images.Ideally, however, one or no network hops are present between the usercomputer 13 and the visualization computer 5. Such a system would existwhere the visualization computer 5 is located at the Internet ServiceProvider (ISP) for the user computer. The user there communicates withthe ISP server and visualization computer over a very short link of adata network, such as local telephone line or a cable TV line, linkingthe user computer 13 with a visualization computer physically in oradjacent to, and wired to, the ISP server.

[0061] As one example of such a system, the user computer 13 comprises avery simple computer system that has the limited function of processinginput from a joystick control thereon and transmitting instruction dataderived therefrom over cable TV wiring to a server.

[0062] The server contains or is wired directly to a visualizationcomputer which renders images from the joystick input data and fromscene data, and then returns rendered images as real-time cable TV videoover the cable TV line.

[0063] According to the preferred embodiment of the invention, thecomputer is equipped with a software package which allows the user tosend control data to create or edit the scene without the need totransmit scene data itself. In other words, the user computer softwarereceives input from one of the input devices thereon, and interprets theinput as instructions or data configured to interface with software orhardware of the visualization computer. This interface results in achange in the scene data. The user computer 13 transmits control data,the interfacing data or instructions, to the image generating system viadata communication connection 25 with the link 15. This control data isconfigured to influence the ultimate rendered image, and comprises datathat is used to control or influence either the scene data itself or theviewing parameter data used by the image generating system to render theimage. Although it is less desirable from the standpoint of reducingdata traffic load and the size of the application running on the usercomputer, the control data may include partial scene data to beincorporated into the scene data on the visualization computer. Thecontrol data may also be one or more instructions to a softwareapplication operating on the visualization computer 5.

[0064] Where the control data is instructions to a software applicationrunning on the visualization system, the instructions may cause any of avariety of changes to the scene data, to the point-of-view data, or tothe viewing parameters data. For example, the instruction may be adirect edit command for a visual object in the scene, such as to changeits color, size, orientation or position. The instruction may similarlydirect the visualization computer software to make a change in the pointof view or the viewing parameter data. The instruction also can be ahigher level instruction to a specialized application, such as to gamesoftware running on the visualization system, which would cause asubstantive change in the scene, such as, for example, opening a door,pushing a rock off a cliff, or any other imaginable action in thevirtual scene. The instruction can also have a combined effect on thescene and viewing data, as, for example, where the instruction is tochange the point of view, and this results in the movement of an avatarof the user in the scene which can be seen by the user himself oranother user.

[0065] Because the user needs only the limited software package tocommunicate with the image system, there is no need for the user toacquire or download a large software package or to obtain specialhardware such as graphics cards to perform the tasks performed by theimage generating system to which the user is linked.

[0066] Preferably all users on the system can access a virtual scene inthe same area thereof and make non-conflicting changes while the otherusers observe. Conflicts in user-directed changes are processed byconventional methods of task sharing of data, which are well known inthe art. In addition, a lockout function in the geometry system softwareprevents two users simultaneously changing the same object in the scene.

[0067] The scene data and the viewing parameter data are stored in theimage generating system both in the visualization systems 7 and thegeometric computer systems. However, since each visualization system 7generates electronic images for only its user(s), the visualizationsystem does not require the scene data in its entirety, but only thatportion of the scene data that is relevant to the point of view of theassociated user(s). The geometric system 3 therefore preferablytransmits only relevant up-to-date scene data to each visualizationcomputer 7 at a high frequency, and ensures that the subsets of scenedata that are sent to the visualization computer 7 are the most currentversions of those subsets, as the scene data is modified by the actionof the many users of the system. Because the voluminous scene data isstored only in the image generating part of the overall system, the usercomputers need not support the large amount of memory or storage spaceneeded to accommodate complex scene data.

[0068] The viewing parameter data defines how the scene is viewed, andincludes, at a minimum, data that identifies a point of view of the userwhich the visualization computer 5 uses in rendering the electronicimage. Subsequent changes in the point of view can also be entered bythe data input devices, which may be the keyboard 27, the mouse 29, ajoystick (not shown), or any of a variety of other input devices wellknown in the art The viewing parameters may also include parameters thatdefine the optics of a camera, which is simulated in the rendering ofthe view of the scene. Such parameters can be focal distance, depth offield, distortions due to camera lenses, or other general parameterssuch as graininess, filtering, etc.

[0069] Each visualization computer may support one or more usercomputers over the associated links 15. Consequently, the computationaldemands on a given visualization computer may be quite high whileanother visualization computer linked thereto by data link 15 is idle orless heavily loaded. Accordingly, the visualization systems are providedwith load-sharing software that allows visualization systems to sharethe computational effort of rendering one or more images for a givenuser. The data processed and the resulting data are transferred betweenvisualization computers by direct or indirect links between thevisualization computers, or possibly via the high-speed links 9 of thegeometric system 3. Algorithms for this type of load sharing are knownin the prior art, and generally rely on running assessments ofindividual computer computational load with protocols for balancing thecomputation.

[0070] Similar load balancing of computation is also provided among thevarious geometry computer systems 7 to efficiently distributecomputations involving the scene data. Since the geometry computers 7are fully connected, directly or indirectly, to all other geometrycomputers by links 9, computational load can be readily distributed andbalanced throughout geometry control system 3.

[0071] Streaming video images may be generated together with streamingaudio. The audio portion of such a signal is generated by a sound enginehardware or software on the visualization computer. The resulting soundis produced based on the scene data, and possibly other sound recordingfiles and is made audible over an audio system of the computer. Thesound engine preferably provides a realistic acoustic effect, such asreverberation, echoes, etc., based on the scene data.

[0072] Similarly, the user computer may be fitted with tactile outputdevices that simulate the feel of an object in the scene. Devices ofthis type are currently available on the market, and these devices cansimulate a variety of textures, such as sandpaper or ice.

[0073] In an alternate embodiment, shown in FIG. 2B, the imagegenerating system may provide rendered images to user computers 13 as asingle frame at a time, with a turnaround time which is slower than areal-time interactive system. The user computers 13 are similar to thoseof the preferred embodiment, although slower communication software andhardware may be used therewith. If a change to the scene is made by anyof the user computers 13, the control data reflecting these changes istransmitted to the image generating system 30 over links 32 via acomputer data network 34. The image generating system 30 automaticallyreturns the rendered electronic image to the user computer 13automatically responsive to the control data.

[0074] In such a system, a turnaround time for returning an image can beas much as 5 to 10 seconds, so the latency issue addressed in thereal-time interactive system above with its geographical distributiondoes not arise. Consequently, the image generating system 30 of thealternate embodiment preferably combines both the geometric andvisualization systems 3 and 7 in a single location and can be located atany distance from the users, preferably on a server computer systemconnected directly to the data network, which is preferably theInternet. Many users can access the visualization system, and theirtasks are simply run in a queue.

[0075] The method by which any of the image generating systems rendersan electronic image can be virtually any rendering system compatiblewith the turnaround tine needed for the given application. For example,the visualization system can use graphics pipeline circuitry, such asOpen GL cards, to create the image and then transmit the image to theuser.

[0076] Particularly preferred, however, is the improved rendering systemdisclosed herein, which provides especially fast rendering, particularlybeneficial in a real-time interactive system.

[0077] b. Organization of the Scene Data One of the features of thepresent system that is preferably used to enhance the speed of renderingis the organization of the scene data in a database stored in a computerdata storage device or memory that allows rapid identification of thevisual objects that are contacted by a ray or beam tracing through thescene space.

[0078] The scene data defines the light sources and objects in thethree-dimensional scene. Light sources are defined as eitherparallel-ray directional lighting (such as sunlight or moonlight), pointlighting, or as area light sources, which are two- or three-dimensionalsurfaces defined by one or more triangles or quadrilaterals. The objectsin the scene are defined as visual objects meaning anything thatreflects or alters light that contacts it, including transparentobjects.

[0079] Visual objects may be defined by any protocol known in the art.Visual objects are generally triangles (also referred to as primitives),tesselated surfaces (i.e., assemblies of triangles), or higher orderobjects, e.g., spheres, ellipsoids, or more geometrically complexsheets, such as NURBS surfaces or patches, which are non-planar surfacesdefined by a function or a combination of functions. The triangularprimitives are identified by the coordinates of their three vertexpoints. Higher complexity objects are defined by the function thatdescribes them, together with whatever point or points are appropriate.Also, where a visual object is a complex surface, the definition of theobject also includes a normal vector to the surface of the object,defined either numerically or as a function.

[0080] Each visual object also has a set of additional parameters whichfurther define its visual characteristics, e.g., a color, a transparencycoefficient, a specular coefficient, a texture definition, and areflectance identifier. The visual object may also be a layered objecthaving a plurality of these parameters each representing a layerthereof.

[0081] The color of the object is preferably defined according to acommon computer color definition in which three numbers represent,respectively, the red, green, and blue intensities of the color. Thetransparency coefficient defines how much light can pass through theobject. This transparency coefficient ranges from 0 to 1, with 0corresponding to complete transparency, and 1 corresponding to totalopacity. The specular coefficient is also a number from 0 to 1 whichdefines the specular quality of the visual object, i.e., how mirror-likethe surface is, with 0 indicating a completely diffuse surface and 1indicating a mirror surface.

[0082] The texture is preferably defined by a code or tag pointing to astored bitmap or image that can be mapped onto the object to give it anappearance of concrete, marble, sand, etc. The textures may be one-,two- or three-dimensional arrays. Alternatively, the texture may bedefined as procedural texturing, wherein a formula or algorithm isidentified or defined which, when calculated or performed, generates thedesired texture for the object.

[0083] The reflectance identifier is a code or tag that specifies whichbidirectional reflectance distribution function (“BRDF”) should be usedfor the surface of the object. The specific BRDF defines how the surfaceof the visual object scatters light as a function of the angle of theincoming light and the angle of viewing. The BRDF will be discussed ingreater detail below with regard to rendering.

[0084] The scene data may be created in a variety of ways. For one, itmay be created by a user through one of the user computer systems, inwhich case, the user computer and/or the visualization computer isequipped with a communication software package which enables the user togenerate and edit visual objects in the scene data stored in the imagesystem. A number of appropriate interactive programs exist on the marketwhich offer this function, including 3D Studio Max from AutoDesk, Inc.Alternatively, the scene data can be created by an application developeron one of the visualization computers, on one of the geometry computers,or on a separate system and downloaded into the geometry control system.

[0085] The defined visual objects may be downloaded from the usercomputer to be placed in the scene, or they may be transmitted in morecompact format. A particularly effective reduction in the size of thetransferred visual objects during creation or editing thereof isobtained by defining the texture of an object procedurally, i.e.,defined as a function as opposed to a bit map. When the user defines atexture by a function, this function is transmitted to the imagegenerating system instead of transmitting the bit map for the texture.The data defining a function for a texture such as marble or wood grainis generally small in comparison with the data for a color bit map ofthe actual texture. The transmission of the selected texture of anobject by its functional definition therefore represents a substantialreduction in the amount of data that must be transferred from the usercomputer to the image system.

[0086] During or after creation of a scene by defining the various sceneelements, it is advantageous according to the present invention toorganize these scene elements in data storage so as to make it easier torender an image from the scene data. This is accomplished using adatabase having a data structure that organizes the scene for moreefficient storage and transmission, and, more importantly, for moreefficient calculation of relevant scene elements to reduce thecomputation time needed for rendering the scene.

[0087] The pre-rendering organizing process is illustrated by an exampleshown in FIGS. 3 and 4. In FIG. 3, a very simple scene is shown,comprising a set of visible articles 35 composed of visual objects anddistributed in the scene space 33. FIG. 4 shows the same scene in afront elevational view.

[0088] To organize the scene data for the scene database, the entirescene volume, sometimes referred to as “the world” or the “theuniverse”, is initially partitioned into a grid of cells, which arepreferably cubic or rectangular boxes with sides at right angles to eachother. These cells are preferably identified by a Cartesian-typecoordinate system, with each cell identified by three non-negativeinteger coordinates. The world space can be divided into as many gridcells as are desired.

[0089] For simplicity's sake, in the example of FIG. 3, the world orscene 33 is partitioned by midpoint bisecting cuts A and B into eightequal subdivisions or cells or boxes, four of which, 37 a, 37 b, 37 c,and 37 d, are seen in FIG. 4. It will be understood that anotherbisecting cut is made to partition the front half of the scene 39 fromthe back, although that cut and the four rearward cells are not visiblein the view of FIG. 4.

[0090] After the scene is divided into the grid of cells, each of thecells is then analyzed by counting the number of visual objects that arefound in it. If the cell is empty, or if it contains fewer than asubdivision threshold number of visual objects, usually a number between10 and 100, no further analysis or subdivision is performed on thatcell. If, however, the cell has more than the subdivision thresholdnumber of visual objects therein, it is subdivided further by divisioninto two along each dimension. Alternatively, subdivision into a numberof cells greater than 2 in each dimension, i.e., a subdivision number of3 or more, may also be used, instead of subdivisions by 2's. Subdivisionin 2's is preferred, however, because it gives a computational advantagein a binary digital computer, where multiplication and division by 2 isa bit-shift operation.

[0091] Furthermore, where the individual box contains a large number ofvisual objects, e.g., more than 10,000, and these are distributedthrough the box, it is most probable that this box will be subdividedseveral more times before the count of visual objects in each sub-box isless than the threshold. Recording the repeated subdivision of the boxesis a waste of memory, and also makes processing of the scene datainefficient. Accordingly, when a cell contains a number of visualobjects that is larger than a multiple subdivision threshold number,that cell is subdivided not by the subdivision number (i.e., 2) ofpartition in each dimension, but by an integer power of the subdivisionnumber, here 2.

[0092] This process is illustrated in FIG. 4. The space 33 is subdividedonce into cells or boxes 37 a, 37 b, 37 c, and 37 d, and four boxesbehind them that cannot be seen. Box 37 b contains no geometry, so it isnot subdivided any further. Boxes 37 a and 37 d contain some geometry;they are subdivided by two partitions in each dimension. Assuming forthe sake of example that each of the sub-boxes 39 a to 39 h has fewerthan the subdivision threshold number of visual objects, they are notsubdivided further.

[0093] Sub-box 37 c, however, contains a large number of visual objectswhich exceeds the multiple subdivision threshold, and these aredistributed throughout the box 37 c. This sub-box 37 c is thereforesubdivided in eight (8) sub-cells in each dimension, yielding 83 (i.e.,512) sub-cells 41. Assuming, again, for the sake of the example, each ofthese sub-cells 41 contains fewer than the subdivision threshold numberof visual objects, these cells 41 are not further subdivided. However,if the number of objects in any cell 41 exceeds the subdivisionthreshold number, subdivision of that cell 41 is repeated until nosub-cell therein contains more than the subdivision threshold number ofobjects.

[0094] The number of multiple subdivisions taken may be any number whichis an integer power of the usual subdivision number. In the preferredembodiment, the subdivision number is 2, so the multiple subdivisionsare ₂M in each dimension, yielding ₂₃M subcells. The number M ofmultiple subdivision levels preferably depends on the number of visualobjects in the box being subdivided. Preferably, the multiplesubdivision number M is calculated as directly proportional to thequantity:

log_(q)(N/T)

[0095] where N is the number of objects in the cell, T is the singlesubdivision threshold (the minimum number of objects in a cell totrigger a single subdivision), and q=s³, where s is the usualsubdivision number which in the preferred embodiment is 2.

[0096] Where a large number of objects are present, if the objects arebunched in a small region of the box, only single subdivision, notmultiple subdivision, is used.

[0097] This type of scene data structure is referred to as amultiresolution virtual uniform subdivision (MVUS) data structure. Thevirtual subdivisions are called uniform because, due to the multiplesubdivision number being an integral power of the subdivision number,the planes of subdivision of coarser levels of resolution always line upwith a plane of subdivision of those cells that are divided in a finergrid. This can be seen in FIG. 4, where partition plane C in box 39 caligns with partition plane D in multiply subdivided box 37 c.Similarly, plane E in box 37 d aligns with partition plane F. Thespatial result of this is that each face of each box faces an integralnumber of adjacent box faces, which is advantageous for tracing rays orbeams therein, as will become herein apparent.

[0098] As best shown in FIG. 5, for efficient storage, the scene datathat has been organized in an MVUS structure is loaded into a hash table43. The hash table has m positions, each of which may have one or morerecords 45 therein, or no record. Only those boxes or cells that containgeometry are stored in the hash table 43, so if a cell is not found inthe hash table 43, that cell is empty.

[0099] The individual record 45 contains a number of packets ofinformation. At the very least, the record of a cell that has geometryin it contains coordinate fields i, j, and k which identify the box byits location in a Cartesian coordinate system of the MVUS structure. Thefield l contains a number identifying the level of subdivision of thecell. For example, in FIG. 4, for box 37 b, l=0. For boxes 39 a to 39 h,l=1. For boxes 41, l=3.

[0100] Data field n stands for the number of further subdivisions of theparticular cell. Where n is zero, this means that the box is notsubdivided further. Where n is 1, as it is for boxes 37 a and 37 d ofFIG. 3, there is one further level of subdivision. For box 37 c, n is 3.

[0101] The Cartesian coordinate system for the MVUS sub-cells is bestillustrated in a two-dimensional scheme in FIGS. 6 to 8. Those ofordinary skill in the art will be able easily to appreciate theextension of the simplified 2-D example to a 3-D space.

[0102] Each box is identified by the coordinates of the corners of thebox which are lowest in value. At each level of subdivision, thecoordinates are multiplied by the subdivision. Thus, in FIG. 6, the fourlevel-zero cells are identified by indices i and j as (0,0), (0,1),(1,0) and (1,1). It will be understood that for a more complex scene,the level-zero grid usually contains a much larger number of grid cells,and the set of coordinate values will be from (0,0,0) to the number ofcells in each dimension, e.g., (1000, 800, 100).

[0103] At a single further level of subdivision, as best seen in FIG. 7,the coordinate scale of the lower-resolution level is multiplied by 2 toyield the coordinates of the smaller higher-resolution sub-cells socreated. This multiplication by 2 is particularly efficient in a digitalsystem because, in binary, this is accomplished by a simple bit shift tothe left one place. In the example shown, the coordinates of the level 1boxes are (2,2), (2,3), (3,2) and (3,3). For multiply-subdivided boxesat level 3, as seen in FIG. 8, the coordinates are multiplied by 2⁴=16.The indices of the level 3 boxes range from (8,0) to (15,7).

[0104] The location of the stored data record in the hash table for agiven box identified by its three spatial coordinates i, j, k, and byits level number l is determined by the formula:

hash address=(c_(i)i+c_(j)j+c_(k)k+c_(l)l)mod_(m)

[0105] where$C_{i} \approx \frac{C_{j}}{10} \approx \frac{C_{k}}{100} \approx \frac{C_{1}}{1000} \approx \frac{m}{10,000}$

[0106] The resulting hash table allows for ready access to the scenedata in a given cell.

[0107] The scene data is obtained using the data as shown in FIGS. 9 and10. The hash table record 45 of an undivided cell, i.e., an end node ofthe MVUS subdivision, comprises the i, j, k, 1, and n fields, asdiscussed above, with n being zero. After the n data field, the recordcontains data identifying the visual objects in the cell.

[0108] In the simplest type of scene data arrangement, the dataidentifying the objects is a pointer identifying a location in aseparate stored database which is a list of visual objects organized bycell. In the preferred embodiment, however, several ways of defining thecontents of a cell are available, including pointers to objects in avisual object list, but also data identifying oriented nested objectsformed of many visual objects.

[0109] Accordingly, in the preferred embodiment, the hash table record45 includes a scene graph pointer field 47, a series of object pointerfields 49, and a nested MVUS pointer field 51.

[0110] Object pointer fields 49 each contain a pointer to a location ina separate visual object database 53 of visual objects in the scene. Thelocation in the visual object database 53 holds a record 55 defining avisual object which is defined as has been mentioned previously.

[0111] The scene graph pointer field 47 contains a pointer thatidentifies a node of a scene graph data structure 57 (see FIG. 9) of astored database of visual objects. The scene graph 57 has a number ofnodes 59 each of which holds a record. The node record 59 includestransform matrix data 61 and a virtual instance pointer field 63. Theterm “virtual instance” refers to an occurrence in the scene of anobject which is a fairly complicated assembly of visual objects thatoccur repeatedly in a scene, such as, for example, a seat in a stadium.Rather than store many copies of the data defining the visual objectsthat make up the repeated object, a single copy of the data defining theobject is stored in a library of objects 67, and this is used wheneverthere is a virtual instance of the object. The transform matrix is usedto transform the coordinates of the object as defined in the libraryinto the actual coordinates of the particular virtual occurrence in thecell.

[0112] The light list 65 identifies the light sources in the database ofvisual objects 53 that are relevant, i.e., that shine light onto objectsin the cell. The light list also identifies any parallel beam light inthe scene by the vector of its direction of propagation.

[0113] The nested cell pointer field of the hash table record contains apointer to the beginning of a segment of a nested cell structuredatabase. A nested cell is used when a complex object comprising asubstantial number of visual objects occupies only a small portion ofthe box it is in. In such a situation, illustrated in FIG. 11, most ofthe cell 69 is empty, except for the object 71. A box or boundary volume73 is defined by an algorithm that identifies the object 71 andcircumscribes the object 71 with a box, which is then rotated and shrunkin to fit closely around the object 71. This boundary volume 73 is thenparsed using the MVUS procedure, with subdivision partitions applied asto a larger scene as was discussed above. The nested MVUS hash table isstored coupled with data defining a transform matrix that expresses theorientation of the nested box relative to the larger box and that isused to transform a ray or beam in the larger box into the internalcoordinates of the nested boundary volume 73. Once the ray or beam isconverted to the coordinate system of the nested MVUS, the nested MVUSitself is traversed exactly as the larger surrounding MVUS, and a nestedMVUS may contain further nested MVUS boxes with respectivetransformation matrices which are processed similarly.

[0114] C. Rendering the Electronic Image

[0115] In the preferred embodiment, the image generating system createspixel data which constitutes a view of the scene defined by the scenedata, in the MVUS data structure as seen from the point of view of theuser as specified by viewing parameters data. The MVUS data structureallows for very fast determinations of ray tracing, and also beamtracing, to identify what visual objects are visible from a given pointof view during rendering.

[0116] To illustrate the rendering method of the preferred embodiment, aview of a three-dimensional scene to be rendered is shown in FIG. 13. Itwill be understood that the illustration is only a graphicalrepresentation of the computations performed by the rendering system, asthe objects of the scene are in reality mathematically defined byelectronically stored data.

[0117] The scene space 77 encloses a number of opaque and transparentvisual objects. An opaque chair 79 is adjacent a transparent glass table81 on which an opaque magazine 83 is resting. The scene is illuminatedby a single rectangular light source 85 in the ceiling 87. The image tobe rendered is from point of view 89, with the direction of viewingdefined by normal vector 91.

[0118] The image for which pixels are to be derived is delineated by aframe 93 which defines therein a grid of pixels of the desiredresolution for the image. For each pixel, at least one ray andpreferably between 4 and 64 rays are extrapolated from the point of view89 through the pixel and into the scene space 77. The directions ofthese rays are governed by the viewing parameters data, and theirpurpose is to determine which point of which visual object in the sceneis seen from the point of view 89 through the specific pixel. Once eachray's contact point is determined, the rendering process determines whatcolor it will appear to have, and then assigns the pixel that color, or,if more than one ray is used, combines the colors of the several raysthrough the pixel, and assigns the averaged color to the pixel.

[0119] The process of tracing a ray in an MVUS data structure isillustrated in FIGS. 12A and 12B.

[0120] The first step 94 in a ray trace is to initialize the data areasused in the trace. The priority queue is emptied, and the nearestcontacted object is set to infinity, meaning that any object contactedby the ray will replace it. If the ray does not strike the “world MVUS”,i.e., the largest MVUS of the scene (decision 95), the process moves toA (finished). If a collision object has been found at finish, thatobject is returned as the object struck by the ray. Otherwise, a zero(black) or the MVUS background color is the returned value. Thebackground color value may also be a portion of a bit map of the imageof the wall of the MVUS grid which the ray hits, such as a view ofmountains in the distance or a flat representation of an adjacent scene.

[0121] If the ray does hit the world MVUS, then the first cell ofcontact is calculated and the world MVUS is listed in the priorityqueue. Control then passes to the decision block 97 for queueprocessing.

[0122] At the first pass, the queue holds only the world MVUS. Insubsequent passes, the queue is loaded by loop iteration 98 with all theMVUS's in the current cell. In either case, the queue is checked forentries. If empty, the process goes to A and the currently identifiednearest collision object is returned as the contacted object. If thequeue is not empty, the nearest MVUS in the queue is compared with thecollision object. If the nearest collision object is nearer (decision99), the currently identified nearest collision object is returned asthe object struck by the ray.

[0123] If the MVUS is nearer, it is traversed by a traversal subroutine100, which is detailed in the flowchart of FIG. 12B.

[0124] In the subroutine 100, the first contacted cell is identified,and the i, j, k, and l coordinate values of the contacted box are thenused to calculate the hash address for the MVUS hash table. If the hashtable does not have an entry for the specified cell (decision 101), andthe lower resolution cell is not known (decision 102), then thetraversal pulls back to a lower resolution of the MVUS grid, andcontinues to pull back until a cell in the hash table is found or thelowest resolution of the MVUS is reached (a simple bit-shiftingoperation where the subdivision number is 2). If the lowest resolutioncell is reached, decision 103 checks if the ray has left the MVUS anddecision 104 checks if a preset distance limit on the distance of travelfor the ray has been reached. If either is the case, the traversalsubroutine returns. If neither is the case, the ray is extended to thenext cell of the MVUS, and that cell is checked for presence in the hashtable (decision 101).

[0125] If the current cell is in the hash table, it is checked whetherits geometry is at a higher resolution. If so, the process shifts to ahigher resolution, i.e., to smaller cells, with the current cellcoordinates adjusted by the next level of subdivision of the currentcell (again a simple bit-shifting operation where the MVUS subdivisionnumber is 2), and the hash table is checked (101). If the geometry is atthe present resolution, the visual objects are put in a list to bereturned to the main routine (105). The ray is checked for whether ithas reached the end of the MVUS or whether the predetermined maximumdistance has been reached. If not, the next cell that the ray contactsis set as the current cell, and the subroutine returns. Otherwise, thesubroutine 100 returns with the same cell as current cell.

[0126] On return to the main routine, the ray is checked for whether ithas left the nearest MVUS (decision 106). If it has, that MVUS isremoved from the queue (107). Then, after B, if geometry was returned bythe traversal operation, a collision check is performed (108). If theray collides with an object (decision 109), that object is made thenearest collision object if it is closer than the presently identifiednearest object (110). Control then proceeds to C, where the current cellis scanned for any nested MVUS's that the ray hits (iteration 98).

[0127] The result of the process of the flowchart is the identificationof the visual object in the scene that is struck by the ray.

[0128] The determined contacted object is then checked for transparency.If the object is transparent, its light filtering attributes areincorporated in a queue of transparent objects through which the raypasses, and the ray is traced and extended through the visual object asbefore, until it strikes another object, which is in turn examined foropacity and specularity, and processed as was the first.

[0129] If the point struck by the ray is on an opaque object, the colorof that point is the color of the ray for determining the pixel imagedata. The color of a point is derived as the sum of two color values,(1) the color of light reflected from the point from direct illuminationfrom light sources in the scene, plus (2) the color of light that isindirectly illuminating from the point from other illuminated objects.

[0130] The color of the reflected direct illumination is derived fromthe indicated BRDF for the visual object, the angle of viewing of thevisual object at the point, and the angle of incident light at thepoint.

[0131] The example of FIG. 13 illustrates the process for determiningthe color of reflected direct illumination for a point on an opaquevisual object. One of the rays used in the rendering, ray 95, is shownextending through a pixel in frame 93 and contacting the seat of chair79 at point 97. The sole source of illumination for point 97 is lightsource 85 in the ceiling 87 of the scene 77. To assess how much lightarrives at point 97 from this light source 85, a beam trace is madeusing a pyramidal frustrum beam generally indicated at 99, starting atpoint 97. This beam trace is used, in combination with graphics pipelinecircuitry which is “tricked” into doing the calculations, to generate araster pattern which represents the light source 85 as seen from point97. The pipeline graphics hardware in the disclosed embodiment ispreferably Open GL accelerator chips made by NVIDIA, but more efficientsystems can be envisioned which are specifically designed and dedicatedto this particular process.

[0132] The frustrum 99 is defined by four expanding edge lines 111, 113,115 and 117 starting at point 97. The frustrum 99 is pyramidal, with aquadrilateral, preferably square, cross-section, as indicated bycross-section 119. The length of the side of the square of thecross-section of frustrum 99 in the area of the light source 85 is atleast the maximum diagonal dimension of the quadrilateral light source85 as seen from point 97.

[0133] As the frustrum is being defined, random angular orientation isimparted to the frustrum 99. The benefit of this random rotation may beseen in FIGS. 23 and 24. FIG. 23 shows a rendering wherein the lightsources are calculated using a 16×16 rasterized light sourcecalculation. With a uniform orientation of the frustrum, the rasterizedrepresentation of the light source intensity results in aliasing of thelight intensity, which is ultimately visible as a rippling in theshadows. In contrast, FIG. 24 shows an image rendered using a 32×32rasterization of the light source with a random rotation applied to thefrustrum, resulting in a uniform shadow without ripples.

[0134] The rasterizing of each light source preferably makes use ofgraphics pipeline circuitry, taking advantage of certain computationalsteps that these circuits can perform very efficiently. Pipelinegraphics circuits perform basically two types of functions very rapidly:(1) combining a raster image with a bit map, a process sometimesreferred to as texture mapping; and (2) converting mathematicaldefinitions of triangles into pixels (usually referred to as scanconversion). The present invention takes advantage of both these specialcapabilities.

[0135] As best seen in FIG. 14, the hardware of the visualization systemincludes a digital portion 121 which receives the control data from, andtransmits the image to, the user computer, as has been discussed above.The digital portion 121 also stores the 3D scene data indicatedschematically at 123. The digital portion 121 is also wired to ahardware circuit 125, which is preferably a circuit comprising a set ofpipeline graphics components, arranged to parallel-process data receivedfrom the digital portion 121.

[0136] The frustrum 99 is defined as described above by software runningon the computer portion 121 of the visualization system. The computerportion 121 projects the frustrum as a beam through the MVUS scene spaceto ascertain what visual objects lie within the frustrum. The processfor extending a beam is similar to that of extending a ray through theMVUS space. For a beam, however, some additional computation is requiredto identify all objects in its volume.

[0137] From the starting point, the apex of the frustrum, the dominantdirection of the beam, i.e., the x, y, or z direction in which the beamproceeds most quickly is determined. The cross-section of the beamperpendicular to the dominant direction becomes the advancing front ofthe beam as it extends through the MVUS space. Every cell of the MVUSthat falls wholly or partly in the beam is opened until the terminalsub-cell is reached, and a list of intersected cells containing geometryis kept as the beam goes through the MVUS structure.

[0138] When the list of cells falling partly or completely in the beamis obtained, the visual objects therein are collected and assembled intoa “mini-scene” consisting only of the visual objects in the frustrum 99and the light source being beam-traced.

[0139] Each light source is in the list of visual objects, and isdefined as one or more triangles or quadrilaterals with three or fourvertex points, an illumination intensity value, and, optionally, apointer to an emission distribution function (“EDF”) that defines thelight source light emission pattern. The EDF is preferably a bit mapthat defines the pattern of the light source emissions. For example, anincandescent light bulb is brighter at its center where the filament isglowing. Similarly, fluorescent light panels usually show bands ofbrightness over the fluorescent bulbs therein. The EDF defines a bit mapthat expresses this variation in intensity.

[0140] The digital portion 121 transmits this mini-scene data to thehardware pipeline 125, together with the light source definition and itsEDF, and a definition of the reflection point, i.e., the point 97 forwhich the amount of incident light is being determined and itsreflective parameters (i.e., its BRDF). From this mini-scene data, thehardware 125 generates a raster light pattern, or, alternatively, asingle luminance value, which is transmitted back to the digital portion121. This raster which represents the direct illumination from lightsource 85 that is reflected from point 97 along the ray 95.

[0141] In the pipeline circuit process performed by the hardware 125,the first step is defining a raster pattern which is a silhouette of thelight source 85 in the frustrum 99 in a pure black and white pattern, asshown in FIG. 15. The raster pattern is preferably fairly coarse, e.g.,16×16 or 32×32 pixels. Greater raster resolution is preferably used,however, where the light source is at a very oblique angle relative tothe cross section of the frustrum.

[0142] After the black and white silhouette is defined, the pipelinecircuit combines the black and white silhouette (FIG. 15) with therelevant EDF, which is a bitmap that is part of the mini-scene data, andwhich is mapped onto the visual object of the light source. Thecombination of the EDF texture and the silhouette results in a rasterpattern, such as is shown in FIG. 16, which represents the view of thelight 85 from point 97 without any intervening obstructions.

[0143] The next step in the creation of the light-source raster patternis that any transparent objects which intervene between the light source85 and the point 97 are incorporated in the raster pattern. In thepresent example, this means the green glass top 81 of the table. Thetransparency of the object is defined as a fraction of the red, green,and blue components of light passing therethrough. Each of the occludedpixels of the raster pattern by this transparency fraction data ismultiplied to yield the raster pattern shown in FIG. 17. Alternatively,the transparency may be a more complex transparency texture, definedprocedurally or as a bitmap, mapped against the raster pattern.

[0144] The next step in the formulation of the light-source rasterpattern is determining the occlusion of the light source by any opaqueobjects, and blacking out any occluded pixels in the raster pattern.Such a raster pattern is shown in FIG. 18, wherein the occlusion of theopaque magazine 83 is blacked out of the raster pattern. The resultingraster pattern represents the view of the EDF textured light source, asviewed from the reflective point through any transparent objects, and/orblocked by any intervening opaque objects. This raster represents thelight incident at the reflective point from the light source.

[0145] Once the incident light raster is created, the pipeline hardware125 combines the incident light raster with the BRDF for the visualobject to determine the light diffusely reflected from the reflectivepoint.

[0146] The BRDF functions are derived and used based on a classicalmodel of diffuse reflection. Classically, reflection is modeled using apolar coordinate system, illustrated in FIG. 19. According to thismodel, a visual object 130 has a reflecting point generally indicated at131, and light L from a modeled light source 133 bounces off this point131 to the viewed line of sight to point 131, such as, e.g., line R. Theintensity and color of the reflected light is expressed as a reflectancevalue which is the fraction of incident light that is reflected alongthe line of sight.

[0147] Some materials are anisotropic, such as, e.g., velvet, meaningthat they have reflectance qualities that change depending on the anglethat the material is rotated about a normal axis N. Isotropic materialson the other hand reflect the same intensity regardless of how they arerotated. As best seen in FIG. 19, to model reflectance of an anisotropicmaterial, a direction reference vector D is defined for the material,and the rotative angle θ_(Light) of the incoming light vector L and therotative angle θ_(view) of the reflectance viewing vector R are used todetermine the intensity of the reflection.

[0148] Modeling of isotropic material is simpler. Effectively, θ_(Light)is considered to be zero degrees at all times, and θ_(View) is the anglebetween the incident light vector and the viewing vector.

[0149] In either type of material though, the azimuth angle θ_(Light) ofthe incoming light vector L and the azimuth angle θ_(view) of thereflected light being viewed are used to model the reflection of light.

[0150] Based on the angles θ_(Light), θ_(View), and θ_(View) (andθ_(Light) for anisotropic materials), the BRDF derives a reflectancevalue ranging from zero (no reflection) to one (100% reflection). Anillustration of a usual distribution of reflectance values for a givenincident light vector L is provided in FIG. 20. The shape of the surfaceshown is defined by the magnitude of the reflectance vector R over allthe polar coordinate directions. Normally, maximum reflectance iscentered around a specular angle or mirror angle M, whereθ_(Light)=θ_(View) and θ_(Light)=θ_(View)-180°. This maximum reflectanceis seen as a bulge in reflectance values KR generally indicated at 135.There is also usually a bulge in values called backscatter orretroreflection generally indicated at 137, where θ_(Light)=θ_(View) andθ_(view)<θ_(Light). For most angles, though, the graph is a generallyhemispherical middle portion 139, representing a generally uniformscattering of light, as in a Lambertian surface.

[0151] The BRDF function as used in the present invention follows thisgeneral model, but is preferably defined as data in the oppositedirection, i.e., as a function of reflectance values for a given line ofsight which vary over the field of various values of θ_(Light) andθ_(Light).

[0152] This function is illustrated in FIG. 21A. A single point of view141 sees a point on visual object 143 along line of sight V. A set ofpolar coordinate data is developed for reflection of light to theviewpoint dependent on the θ_(Light) and (p Light for the incident ray.This data is illustrated as hemisphere 145. Each point on the hemisphere145 represents a color value which is the fraction of white lightpassing through that point which would be reflected off the reflectivepoint at the center of the hemisphere to be seen at the point of view141.

[0153] For example, light arriving along ray L passes through point 147of hemisphere 145. This point 147 is in a general region of thehemisphere 145 in which diffuse reflection is higher, i.e., around themirror angle to the angle V of viewing. The ray therefore falls in amore light-colored spot of the hemisphere, indicating greater reflectionof this particular ray. Rays striking at different points would bereflected at lesser degrees, which is indicated by the darker color ofthe hemisphere at those points. For example, a low angle incident lightray passing through point 147 contacts a darker point on hemisphere 145,meaning that the reflection of light of that ray to point of view 141would be minimal.

[0154] This model also allows for modeling of tinted reflection, sinceeach point on hemisphere 145 is a color value.

[0155] To make the hemisphere 145 more readily usable in the presentmethod, especially where graphics pipeline circuitry is used, the BRDFdata for the point of view is converted not into a hemisphere, but intoa hemicube 151 around the reflective point 153 on visual object 154 asillustrated in FIG. 21B. As with the hemisphere 145, each colored pointof hemicube 153 represents the fraction of a ray of white lightextending through that point and reflecting off point 153 that would bereflected to be viewed along line V.

[0156] In the rendering process, the hemicube 151 is used in thepreparation of the rasterized data for illumination of a point by alight source. The BRDF identifier for the visual object struck by therendering ray, such as the seat of the chain 79 in FIG. 13, identifies aset of hemicube texture maps, each representing roughly one degree ofazimuth of the point of view relative to the visual object viewed. Theangle of viewing the visual object and the BRDF identifier are used toselect the proper BRDF hemicube texture map, and the resulting texturebitmap sheet is included as part of the mini-scene. When an anisotropicBRDF is required, both the azimuth and rotative angle of viewing areused to index into a set of textured BRDF hemicube maps to obtain theproper bitmap.

[0157]FIGS. 22A and 22B show how this sheet is formed. FIG. 22A is thesurface of the hemicube of FIG. 21B unfolded to lie in a plane. FIG. 22Bis the organization of the parts of the unfolded hemicube as they aretransmitted to the pipeline graphics circuitry. The format herein isparticularly suitable for OpenGL graphics cards.

[0158] In the miniscene data, the hemicube is assembled around thereflecting point, as in FIG. 21B. The frustrum 99 for the raster imageof the light source extends through the hemicube 151, defining arelevant portion thereof, generally indicated at 155. The graphicscircuitry combines the texture of this part of the hemicube with theassembled raster pattern of all other components of the miniscene (theraster pattern of FIG. 18) to yield the reflected raster pattern of FIG.25.

[0159] This BRDF combined raster pattern represents the direct lightthat is reflected from the point 97. This raster may be returned todigital portion 121 as pixel data from the pipeline circuitry 125, or itmay be returned as a single data value derived as the sum of all of theraster pixels.

[0160] In addition to reflection of direct light, the rendering processalso takes into account reflection of other objects in the scene. Thecalculation of light reflection from these objects is performed by thedigital portion 121 of the visualization computer. This calculation ispreferably performed based generally on the Phong reflection model,which is well known in the prior art, and is discussed in, for examplePhong, “lumination for Computer Generated Pictures”, Communications ofthe ACM (CACM), Volume 18, Number 6, pp. 311-317, June 1975, which isherein incorporated by reference. The result is combined with the resultof the light source rasterization to yield the color returned by theray. If the object is specular (i.e. specular value not zero), the rayis extended in the mirror angle direction relative to the incoming raystriking the object. This specular ray is then extended onward throughthe MVUS grid to see if it strikes any other objects, either in thecurrent box or outside it. If the ray actually leaves the box, the MVUSmethod discussed above is used to determine whether the ray entersanother box containing visual objects. If the reflected ray hits anobject either in the current box or another box, that object is alsochecked for transparency and specularity, and the process repeats.

[0161] If the object that the reflected ray strikes also is specular,the ray is again reflected at a specular angle and extended to the nextobject that it encounters. This specular reflection and tracing backcontinues for a limited number of reflections, e.g., four or five,because these reflections generally are of reducing intensity andbrightness, and consequently have diminishing impact on the illuminationof the first visual object struck by the ray. Each specular reflectionis recorded as the color fraction of the specular light reflected and iskept as a running multiple of the reflections to this point. Thisreflected color fraction is derived from the BRDF field of the specularobject, where the color of the relevant point on the relevant hemicubeis recorded as the specular reflection fraction for the object.

[0162] Ultimately, the ray will hit a non-specular object, will hit alight source, will leave the scene space, or the number limit onreflections will be reached, and the ray will stop. If the ray hits anobject, the color of the object is determined based on the methoddiscussed above for determining reflection for light from light sources,and this resulting color is multiplied by the running multiple of thereflections, with the resulting product being the determined light colorfor the specular reflection. Similarly, where a light source is struck,the color of the point of the light source is multiplied by the runningreflection multiple, and the result is returned as the reflection at thepoint of other objects. Otherwise, a zero (black) is returned, which hasno effect on the color of the point.

[0163] The color from reflected objects is added to the color ofreflected light sources, and the result is the color for the ray, whichis then assigned as the pixel image data for the pixel in question, andcombined (usually averaged) with other rays through the pixel to providethe color.

[0164] The ray tracing and rendering process is repeated for all pixelsof the desired image, resulting in a realistic image, such as is shownin FIG. 26, the rendered image for the scene of FIG. 13.

[0165] It will be understood that, according to an aspect of theinvention, the rendering of images is preferably done by the aboverendering process and using MVUS scene data. However, the visualizationsystem of the invention may also use a different rendering method. Forexample, an existing graphics pipeline system could be used to renderthe image transmitted to the user. Similarly, a conventionalorganization of scene data, such as a voxel grid or octree structure,can be used.

[0166] In addition, although MVUS scene data databases have herein beenshown used with the foregoing rendering process, it should be understoodthat computer-accessible data storage devices storing MVUS scenedatabases can be used advantageously with other image generationprocesses and systems. Generally, the MVUS design can be usedadvantageously in any image generating system wherein the computer orcomputers extend a path (e.g., a ray or beam) through a scene anddetermine if it contacts any visual objects in the scene, or wherein thecomputer or computers locate objects within some spatial neighborhoodaround a point or location in the scene space.

[0167] For example, some rendering methods rely on a global illuminationprocess to render images of the scene. As an example of such a process,a point viewed in the rendered image is identified and then itsillumination is determined by casting 360 degrees by 90 degrees of raysor beams around the point and determining what visual objects or lightsources those rays encounter. If the ray or beam hits another visualobject, the illumination of that point is determined by the same 360 by90 degrees casting of rays or beams. The process is repeated for anyvisual objects that those rays or beams hit, until eachtraced-back-illumination ray or beam meets a light source or leaves thescene, whereupon the total illumination of the viewed point iscalculated. The casting of the rays or beams and the identification ofthe visual objects that lie therein in such a method can be efficientlyaccomplished using an MVUS scene database.

[0168] Also, a process relying on forward tracing may rely on an MVUSscene database. In forward tracing, each light source for the scene isidentified, and the light projected therefrom is traced along a paththrough the scene. When the path contacts an object, reflected light isprojected from the contacted point, and by repeating this process forall points struck by the illumination from the light sources theillumination of all visual objects in the scene is determined. Again,the MVUS database structure improves the efficiency of identifying thevisual objects in the scene that lie in the paths of the light.

[0169] Still other rendering processes not mentioned here can makeadvantageous use of an MVUS scene database.

[0170] The MVUS database of the preferred embodiment is stored on acomputer-accessible data storage device, as is well known to those ofskill in the art. Preferred for this data storage is random accesscomputer memory (RAM), since it allows for rapid recovery of data by therendering computer system. The RAM may be any such computer memorydevices, including SRAM or DRAM, and the database may be stored in asingle computer system, or in two or more linked computer systems.

[0171] In the preferred embodiment, the MVUS database is stored usingRAM memory, and access to this is accelerated using a contentaddressable memory (CAM) operating as a high-speed cache lookup of scenedata without need to access the hash table. The CAM memory is generallysmaller than the data stored in the hash table 43, so the CAM storesonly the most recently accessed entries in the hash table, operating asa cache, i.e., less accessed data is dropped from the CAM as it fillsup. The CAM is accessed by a content key made up of the values i, j, kand l for the MVUS partition being addressed. Such a CAM enhances thespeed of the access to the scene data by an order of magnitude.

[0172] As a further option, where the computer system has a CAM memorythat is large enough to store all the entries of the MVUS hash table 43,then all of the hash table entries may be loaded into CAM memory. Inthat case, the rendering application program will rely exclusively onthe CAM to look up the MVUS database scene graph pointers and otherpointers in the hash table entry, and the hash table 43 itself may bedispensed with.

[0173] Alternatively, because the MVUS data structure reduces the timeneeded to access the scene data, MVUS databases can be usedadvantageously in slower data storage hardware, such as, e.g., computerdisk drives.

[0174] It should also be understood that the terms used herein should beconsidered terms of description rather than limitation, as those withthis specification before them will be able to make variations andmodifications to the disclosed methods and systems without departingfrom the spirit of the invention.

What is claimed is:
 1. A dynamic media system comprising: an imagegenerating computer system having electronic storage storing scene datafor a three dimensional scene; a user computer connected with the imagegenerating computer system over a data communications link andtransmitting control data to said image generating computer system oversaid link; the image generating computer system receiving the controldata and automatically rendering electronic image data from the controldata and the scene data, said electronic image data representing animage of the scene seen from a point of view, the image generatingcomputer system transmitting the pixel image data to the user over thecommunications link; and the user computer receiving the electronicimage data and displaying the electronic image data at the user computeras a visual image viewable by a user.
 2. The system of claim 1, whereinthe communication link is a computer data network.
 3. The system ofclaim 2, wherein the computer data network is the Internet.
 4. Thesystem of claim 2, wherein the computer data network is a local areanetwork.
 5. The system of claim 2, wherein the computer data networkcomprises a wireless cell communication system.
 6. The system of claim1, wherein the communication link is a cable TV link.
 7. The system ofclaim 1, wherein the communication link is a telephone connection. 8.The system of claim 1, wherein the image generating computer systemcomprises a geometric system storing the scene data.
 9. The system ofclaim 8, wherein the geometric system comprises a plurality of separategeometric computer systems each storing therein a respective version ofthe scene data.
 10. The system of claim 8, wherein the image generatingcomputer system further comprises a visualization system connected by acommunication link to the geometric system and receiving the scene datatherefrom, said visualization system rendering the image from the scenedata received from the geometric system and transmitting the renderedimage to the user system.
 11. The system of claim 10, wherein thecommunication link between the geometric system and the visualizationsystem is a dedicated data link.
 12. The system of claim 1, wherein thecontrol data comprises an instruction selected from the group consistingof instructions directing a change of the point of view, instructionsdirecting a change in an optical parameter of viewing the scene,instructions to modify the scene data, and instructions to applicationsoftware controlling the scene data.
 13. The system of claim 10, whereinthe control data comprises an instruction selected from the groupconsisting of instructions directing a change of the point of view,instructions directing a change in an optical parameter of viewing thescene, instructions to modify the scene data, and instructions toapplication software controlling the scene data.
 14. The system of claim13, wherein the visualization system transmits the instruction to thegeometric system.
 15. The system of claim 14, wherein, where theinstruction is an instruction directing a change of the point of view orof the optical parameter of viewing, the visualization system does nottransmit the instruction to the geometric system.
 16. The system ofclaim 1, wherein the image data is pixel data.
 17. The system of claim1, wherein the image data is transmitted to the user computer ascompressed pixel data.
 18. The system of claim 17, wherein the pixeldata is compressed using a JPEG protocol.
 19. The system of claim 1,wherein the image data is transmitted as a video stream.
 20. The systemof claim 19, wherein the communications link is a cable TV link carryingthe video stream.
 21. The system of claim 19, wherein the video streamis in a raw video format.
 22. The system of claim 19, wherein the videostream is compressed video.
 23. The system of claim 22, wherein thevideo is compressed using an MPEG protocol.
 24. The system of claim 19,and said video stream including audio data configured to be converted toaudible sound by the user computer.
 25. The system of claim 24, whereinthe audio data is derived from the scene data.
 26. The system of claim1, wherein the user computer has software therein providing aninteractive application to the user for modifying the scene data.
 27. Adynamic media system comprising: a geometric control system having datastorage, said data storage holding scene data representative of visualobjects of a three dimensional scene; a plurality of visualizationsystems connected with and communicating with the geometric controlsystem; and a plurality of user computers each communicating with arespective visualization system via a link through a computer datanetwork; each of said user computers having a display for displayingimages and a data entry device for entering data to the user computer;the geometric control system transmitting the scene data to thevisualization system; the user computers each transmitting control datato the visualization system associated therewith; the visualizationsystems each rendering from the scene data a respective electronic imagerepresenting a frame view of the scene from a point of view specific tothe associated user computer; and the visualization systems transmittingthe image to the associated user computer, said user computer displayingthe image.
 28. The system of claim 28, and said visualization systemsbeing separated from one another or the network by at least one node,and each being connected with the respective user computer with lessthan three network nodes.
 29. The system of claim 28, and saidvisualization systems each being connected with the geometric controlsystem by a dedicated data link.
 30. The system of claim 29, whereineach of the visualization systems communicates with at least two of theuser computers.
 31. The system of claim 27, wherein the geometric systemcomprises a plurality of geometric host computer systems connected witheach other, each of the geometric host computer systems communicatingwith at least one of the visualization systems.
 32. The system of claim31, wherein the geometric host computer systems each store therein arespective version of the scene data, said version comprising a dataportion of recent scene data and a data portion of older scene data. 33.The system of claim 32, wherein the recent scene data for the geometrichost computer system corresponds to an area in the scene proximate thepoint of view of the user computers associated with the visualizationsystem communicating therewith.
 34. The system of claim 27, wherein thecomputer data network is the Internet.
 35. The system of claim 27,wherein the computer data network is a local area network.
 36. Thesystem of claim 28, wherein the computer data network is a wireless cellcommunication network.
 37. A dynamic media system comprising: a usercomputer having a connection to a data communication link; a data entrydevice connected with the user computer and providing for a user thecapability to enter data to the user computer; and a display deviceconnected with the user computer and displaying images therefrom; theuser computer receiving input data from the data entry device andforming control data from the input data, said control data beingconfigured to cause a change in data selected from the group consistingof scene data defining a 3D scene from which an image can be desired,and data defining optical parameters used in deriving said image; theuser computer transmitting the control data over the data communicationlink; said user computer receiving electronic image data over the datacommunications network, said electronic image data being produceddependent on said scene data and said control data and representing aview of the scene according to the viewing parameters; and the usercomputer causing the display device to display an output image derivedfrom the electronic image data thereon.
 38. The system of claim 37,wherein the data communication link is the Internet.
 39. The system ofclaim 37, wherein the data communication link is a local area network.40. The system of claim 37, wherein the data communication network is awireless cell system.
 41. The system of claim 37, wherein the datacommunication network is a cable TV link.
 42. The system of claim 37,wherein the data communication network is a telephone connection. 43.The system of claim 37, wherein the control data comprises electronicinstructions configured to be received and acted on by a softwareapplication operating in a computer image system connected with the datacommunication link.
 44. The system of claim 37, wherein the control datais configured to cause a change in position or visual attributes of atleast one visual object which is part of the scene data.
 45. The systemof claim 37, wherein the control data comprises data configured to causealteration in data representing a point of view for the view of thescene.
 46. The system of claim 37, wherein the control data comprisesdata configured to cause alteration in data representing an opticalparameter for the view of the scene.
 47. The system of claim 46, whereinthe optical parameter is selected from the group consisting of focallength, depth of field, camera lens distortion optics and atmospherictransparency.
 48. The system of claim 37, wherein the data entry deviceis a keyboard, a mouse, a joystick or a touch sensitive pad.
 49. Thesystem of claim 37, wherein the electronic image data is pixel data. 50.The system of claim 37, wherein the electronic image data is compressedpixel data.
 51. The system of claim 50, wherein the image data iscompressed according to a JPEG protocol.
 52. The system of claim 37,wherein the image data is transmitted a part of a video stream.
 53. Thesystem of claim 52, wherein the communication link is a cable TV link.54. The system of claim 52, wherein the video stream is in a raw videoformat.
 55. The system of claim 53, wherein the video stream iscompressed video.
 56. The system of claim 55, wherein the video iscompressed using an MPEG protocol.
 57. The system of claim 52, and saidvideo stream including audio data configured to be converted to audiblesound by the user computer.
 58. The system of claim 57, wherein theaudio data is derived from the scene data.
 59. The system of claim 37,and said user computer receiving the data input, transmitting thecontrol data, and displaying the output image at points in time in closeenough proximity for real-time interactive access to the scene data fromthe user.
 60. The system of claim 37, wherein the user computer receivesthe data input, transmits the control data, receives the image pixeldata, and displays the output image within less than about 15 seconds.61. The system of claim 37, wherein the user computer receives the datainput, transmits the control data, receives the image pixel data, anddisplays the output image within less than about 0.05 seconds.
 62. Adynamic media system comprising: an image computer system storing scenedata representing a three dimensional scene and viewing datarepresenting at least one viewing parameter for viewing the scene, theviewing data including data defining a point of view; and acommunication connection linking the image computer system to a datacommunication link; the image computer system receiving control datathrough said communication link from a remote source, said control databeing configured to interact with the image computer system so as tocause alteration in the scene data or so as to cause formulation ormodification of the viewing data; the image computer systemautomatically rendering an electronic image from the scene dataresponsive to receiving the control data, said image data representing aview of the scene dependent upon the viewing data; the image computersystem transmitting the electronic image over the communication link ina format effective to reach the source of the control data and in a formsuch that a viewable image can be derived from the electronic image anddisplayed.
 63. The system of claim 62, wherein the data communicationlink is the Internet.
 64. The system of claim 62, wherein the datacommunication link is a local area network.
 65. The system of claim 62,wherein the data communication link is a wireless cell communicationnetwork.
 66. The system of claim 62, wherein the data communication linkis a cable TV link.
 67. The system of claim 62, wherein the control datacomprises an instruction directing a change in the point of view data.68. The system of claim 62, wherein the control data comprises aninstruction selected from the group consisting of instructions directinga change of the point of view, instructions directing a change in anoptical parameter of viewing the scene, instructions to modify the scenedata, and instructions to application software controlling the scenedata.
 69. The system of claim 62, wherein the image computer system isconnected with a second data link and receives the scene data over thesecond data link.
 70. The system of claim 69, wherein the control datacomprises an instruction selected from the group consisting ofinstructions directing a change of the point of view, instructionsdirecting a change in an optical parameter of viewing the scene,instructions to modify the scene data, and instructions to applicationsoftware controlling the scene data.
 71. The system of claim 70, whereinthe image computer system transmits the instruction over the second datalink in a format configured to be used to affect the scene data receivedby the image computer system.
 72. The system of claim 69, wherein thefirst and second data links comprise a connection through the Internet.73. The system of claim 69, wherein the second data link is a dedicateddata link.
 74. The system of claim 62, wherein the electronic image istransmitted to the user computer as pixel data.
 75. The system of claim62, wherein the electronic image is transmitted to the user computer ascompressed pixel data.
 76. The system of claim 75, wherein the pixeldata is compressed using a JPEG protocol.
 77. The system of claim 62,wherein the electronic image is transmitted as part of a video stream.78. The system of claim 77, wherein the data communication link is acable TV connection.
 79. The system of claim 77, wherein the videostream is in a raw video format.
 80. The system of claim 77, wherein thevideo stream is compressed video.
 81. The system of claim 80, whereinthe video is compressed using an MPEG protocol.
 82. The system of claim77, wherein the video stream includes audio data configured to beconverted to audible sound.
 83. The system of claim 82, wherein theimage computer includes a sound engine which derives the audio data fromthe scene data and the point of view data.
 84. The system of claim 62,wherein the image computer system operates so that between the receivingof the control data and the transmitting of the electronic imagerendered therefrom, a period of time of less than about 15 secondselapses.
 85. The system of claim 62, wherein the image computer systemoperates so that between the receiving of the control data and thetransmitting of the electronic image rendered therefrom, a period oftime of less than about 0.05 seconds elapses.
 86. The system of claim62, and the image computer system storing at least some of the scenedata organized in a multiresolution virtual uniform subdivision datastructure, the image computer system using the multiresolution virtualuniform subdivision data structure to render the electronic image. 87.The system of claim 62, and the scene data including data defining alight source and a visual object; said image computer system renderingthe pixel image data, said rendering including beam tracing the lightsource from a location on the visual object in the scene.
 88. The systemof claim 87, wherein the image computer system further includes graphicspipeline circuitry; said graphics pipeline circuitry generatingrasterized illumination data corresponding to said beam tracing.
 89. Amethod of preparing an electronic image, said method comprising:creating scene data defining a three dimensional scene, said scene dataincluding data defining at least one area light source and at least onevisual object; receiving control data over a data communication link;rendering, responsive to the control data, an electronic image fromscene data, said electronic image representing a view of the scene froma point of view; and transmitting the electronic image over the datacommunication link in a form such that the electronic image can bereceived and displayed as a viewable image.
 90. The method of claim 89,wherein the rendering being based on optical parameter data representingoptical parameters for the view of the scene, the optical parametersincluding data defining the point of view; and the control datacomprising data configured to cause a change in the scene data or tocause a change in the optical parameter data.
 91. The method of claim90, and the control data comprising data configured to cause a change inthe scene data.
 92. The method of claim 91, wherein the control data isan instruction to a software application generating or modifying thescene data.
 93. The method of claim 92, wherein the software applicationis run on a first computer system and the rendering step is run on asecond computer system.
 94. The method of claim 93 wherein the controldata is received at a second computer system, and the method furthercomprises transmitting the control data from the second computer to thefirst computer.
 95. The method of claim 94, and further comprisingreceiving the scene data from the first computer at the second computer.96. The method of claim 89, and further comprising receiving at acomputer system data transmitted from a separate geometric system, therendering step being performed at least in part by the computer system.97. The method according to claim 96, wherein the control data comprisesdata configured to cause a change in the optical parameter data.
 98. Themethod of claim 97, wherein the optical parameter data is stored on thecomputer system, and the data changing the optical parameter data isused by an application running on the computer system to change theoptical parameter data.
 99. The method of claim 89, wherein saidrendering includes tracing a beam from at least one point on the visualobject and deriving for said point rasterized data representing the arealight source as seen from the point.
 100. The method of claim 99,wherein said beam tracing includes defining a beam frustrum enclosingthe area light source.
 101. The method of claim 100, wherein saiddefining of the frame further includes applying a rotational orientationto the frustrum.
 102. The method of claim 101, wherein the rotationalorientation is a random angular displacement.
 103. The method of claim99, wherein the rasterized data is derived using dedicatedraster-generating circuitry.
 104. The method of claim 103, wherein theraster-generating circuitry performs a plurality of independentoperations each on a respective portion of the circuitry.
 105. Themethod of claim 104, wherein, while the rendering is being done byperforming one of the independent operations, other calculations areperformed using the other independent operations.
 106. The method ofclaim 105, wherein the other calculations are part of a process ofderiving rasterized data from a second light source, or for a differentpoint on the same visual object or another visual object.
 107. Themethod of claim 99, and the beam trace including forming subscene datarepresenting a subscene including the area light source and at leastsome of the visual objects.
 108. The method of claim 107, and therasterized area light data is derived using dedicated raster-generatingcircuitry.
 109. The method of claim 108, and said deriving rasterizeddata further comprising providing the subscene data to the dedicatedcircuitry.
 110. The method of claim 109, wherein the raster-generatingcircuitry performs a plurality of independent operations each on arespective portion of the circuitry.
 111. The method of claim 110,wherein, while the rendering is being done by performing one of theindependent operations, other calculations are performed using the otherindependent operations.
 112. The method of claim 111, wherein the othercalculations are part of a process of deriving rasterized data from asecond light source, or for a different point on the same visual objector another visual object.
 113. The method of claim 89, and said creatingof scene data including organizing data representing the area lightsource and visual object in a multiresolution virtual uniformsubdivision data structure.
 114. The method of claim 113, and saidrendering including deriving for at least one point on the visual objectrasterized data representing the area light source as seen from thepoint; and the point on the visual object being identified by tracing aray from the point of view using the multiresolution virtual uniformsubdivision data structure.
 115. The method according to claim 99, anddirectional reflective coefficients of the visual object being stored asreflection texture data; said rendering step including combining therasterized data with an appropriate portion of the reflection texturedata.
 116. The method according to claim 115, wherein the reflectiontexture data is stored as planar texture data representing a surface ofa box with planar walls around the point.
 117. The method according toclaim 107, the scene data being organized at least in part in amultiresolution virtual uniform subdivision (MVUS) data structure; saidsubscene data being derived using the MVUS data structure for raysproceeding from the point to the area light source.
 118. A method forrendering an electronic image representing a view from a point of viewof a virtual scene defined by scene data comprising a plurality ofvisual objects and at least one area light source, said methodcomprising: identifying one of the visual objects, or a portion thereof,which is visible from the point of view, and tracing a beam from thevisual object or portion, said beams enclosing therein the light source;deriving from said beam tracing illumination data defining the amount oflight reaching the object or portion thereof; and determining from theillumination data an amount of light from the area light source that isreflected from the visual object or portion thereof.
 119. The method ofclaim 118, wherein the scene data includes a second area light source,and the method further comprises tracing a beam enclosing the secondlight source and deriving illumination therefrom.
 120. The method ofclaim 118, wherein the scene data is organized in an MVUS data structureand the visual object is identified by traversing the MVUS datastructure.
 121. The method of claim 118, wherein said beam tracingincludes defining a frustrum that encloses the area light source. 122.The method of claim 121, wherein said frustrum has an angular rotationaldisplacement incorporated therein prior to deriving the illuminationdata.
 123. The method of claim 121, and further comprising creatingmini-scene data defining visual objects in the frustrum.
 124. The methodof claim 123, wherein the deriving of illumination data is performed bypipeline graphics circuitry, and the mini-scene data is transmittedthereto.
 125. The method of claim 118, wherein the deriving of theillumination data is performed by pipeline graphics circuitry.
 126. Themethod of claim 125, wherein the illumination data represents a colorvalue.
 127. The method of claim 118, wherein the illumination datarepresents a color value.
 128. The method of claim 118, wherein the stepof determining of reflected light is performed as part of the derivationof illumination data.
 129. The method of one of claims 118 to 128,wherein the determining of reflected light is performed using areflectance function that varies dependent on an angle of incidence andan angle of viewing at the point of light being modeled.
 130. The methodof claim 129, wherein the reflectance function combines a reflectancetexture map representing varying reflectance at the point based on saidangles with the illumination data to produce the determination of thereflected light.
 131. The method of claim 130, wherein the reflectancetexture map is stored electronically in a form corresponding to amapping of the varying reflectance of light on a surface of a boxstructure around the object or portion.
 132. The method of one of claims118 to 131, wherein the amount of reflected light from the light sourceis combined with a color value for other light reflected off the object.133. The method of one of claims 118 to 132, wherein the electronicimage is a pixel data image.
 134. The method of one of claims 118 to133, wherein an illumination texture is mapped onto the area lightsource.
 135. A method of rendering an electronic image representing aview from a point of view of a virtual scene defined by scene datacomprising a plurality of visual objects, said method comprising:identifying one of the visual objects or a portion thereof which isvisible from the point of view; deriving for the object an angle ofviewing thereof; deriving illumination data for light illuminating saidobject or portion, said illumination data including directional dataindicating an angle of incidence of the light on the object and anintensity value; combining the intensity value with a reflectioncoefficient derived from the angle of incidence and the angle of viewingof the visual object.
 136. The method of claim 135, wherein thereflection coefficient is part of a texture map.
 137. The method ofclaim 136, wherein the texture map is structured as a box surrounding areflective portion of a visual object.
 138. The method of claim 137wherein each discrete point of the box represents the amount of lightpassing through said discrete point that would be reflected along a lineof sight.
 139. The method of one of claims 135 to 138, wherein thetexture map is combined with the illumination data by graphics pipelinecircuitry.
 140. The method of one of claims 135 to 139, wherein theillumination data includes a raster pattern.
 141. A method for renderingan electronic image representing a view of a virtual scene as viewedfrom a point of view, said method comprising: providing scene datadefining the virtual scene in a grid format for efficient tracing of thepassage of light therethrough; and tracing a path for light in saidvirtual scene using said scene data.
 142. The method of claim 141, andsaid path of light that is traced being a beam.
 143. The method of claim141, and said beam being a generally pyramidal frustrum.
 144. The methodof claim 141, and said path being a ray.
 145. The method of claim 140,and said grid format comprising an MVUS data structure.
 146. The methodof claim 145, and said MVUS data structure including a subdivision ofthe virtual scene, said cell subdivision containing at least partiallythereon a pointer to a second MVUS structure.
 147. The method of claim146, and said second MVUS structure having a transformation functionassociated therewith based on a relative orientation of the second MVUSto the first MVUS.
 148. The method of claim 146 or 147, wherein thesecond MVUS is in a library of objects for placement in virtualinstances.
 149. The method of one of claims 141 to 146, said gridstructure containing cells being subdivided into subcells in each ofthree dimensions by a subdivision number where the cell contains anumber of visual objects that exceeds a first subdivision thresholdvalue.
 150. The method of one of claims 141 to 149, wherein the gridstructure has cells which are subdivided into subcells in each of threedimensions in a number which is an integer power of the subdivisionnumber where the cell contains a number of visual objects that exceeds asecond multiple subdivision threshold value.
 151. The method of one ofclaims 141 to 150, wherein cells of the grid structure are recorded in ahash table, said hash table containing said cells based on addressescalculated from grid coordinates of the cell or subcell and a levelnumber representing the level of subdivision thereof.
 152. The method ofone of claims 141 to 151, wherein said tracing is performed using analgorithm as disclosed herein.
 153. An apparatus performing the methodof one of claims 118 to 151, said apparatus including a digitalcomputer.
 154. The apparatus of claim 153, said apparatus furthercomprising pipeline graphics circuitry.
 155. A computer-readable storagedevice storing thereon instructions causing a computer to perform themethod of one of claims 118 to
 151. 156. A method for storing in acomputer-readable database scene data comprising electronic visualobject data defining visual objects each having associated therewithelectronic data identifying a location of the visual object in a virtualspace, said method comprising: a) determining from the electronic visualobject data a number representing a count of the how many of the visualobjects have locations within a cell in the virtual space; b) comparingsaid number of visual objects with a predetermined threshold subdivisionvalue and/or a predetermined threshold multiple subdivision value; c)responsive to a determination that the number of visual objects in thecell exceeds the predetermined threshold subdivision value but not thepredetermined threshold multiple subdivision value, performing asubdivision operation representing a partitioning of the cell in atleast one dimension thereof into an integer subdivision number s ofsubcells that are equal in size to each other and together constitutethe cell; d) responsive to a determination that the number of visualobjects in the cell exceeds the predetermined threshold multiplesubdivision value, performing a multiple subdivision operationrepresenting a partitioning of the cell in at least one dimensionthereof into a number of sub-cells, which number is an integral power,s^(n), of subdivision number s, said sub-cells being equal in size toeach other and together constituting the cell; e) responsive to adetermination that the number of visual objects in the cell does notexceed the predetermined threshold subdivision value, storing dataidentifying the visual objects in the cell in the database so that thevisual object data of said visual objects can be accessed by a computerbased on coordinates identifying said cell; and f) repeating steps (a)to (d) for each of the sub-cells created in steps (c) or (d), whereineach sub-cell becomes the cell in step (a) for which the number ofvisual objects is counted.
 157. A method according to claim 156, whereinthe operations representing a partitioning of the cell are ofpartitioning the cell in at least two dimensions thereof into s or s^(n)sub-cells that are equal in size to each other and together constitutethe cell.
 158. A method according to claim 156, wherein the operationsrepresenting a partitioning of the cell are of partitioning the cell inthree dimensions thereof into s³ or s^(3n) sub-cells that are equal insize to each other and together constitute the cell.
 159. A methodaccording to claim 156, wherein the subdivision number is
 2. 160. Amethod according to claim 157, wherein the subdivision number is
 2. 161.A method according to claim 158, wherein the subdivision number is 2,and where the subdivision operation yields eight sub-cells constitutingthe cell, and wherein the multiple subdivision operation yields 2^(3n)sub-cells of the cell.
 162. A method according to claim 156, whereinsaid cell is identified by integer coordinates thereof, and eachsub-cell thereof is identified by at least one coordinate derived bymultiplying at least one of the coordinates of the cell by thesubdivision number to obtain a product, and adding a non-negativeinteger number less than the subdivision number to the product as anindication of displacement from an edge of the cell.
 163. A methodaccording to claim 157, wherein said cell is identified by integercoordinates thereof, and each sub-cell thereof is identified by at leasttwo coordinates derived by multiplying at least two of the coordinatesof the cell by the subdivision number to obtain products, and adding arespective non-negative integer number less than the subdivision numberto each product as an indication of displacement of the sub-cell fromedges of the cell.
 164. A method according to claim 158, wherein saidcell is identified by three integer coordinates thereof, and eachsub-cell thereof is identified by three coordinates derived bymultiplying the coordinates of the cell by the subdivision number toobtain three products, and adding a respective non-negative integernumber less than the subdivision number to each product as an indicationof displacement of the sub-cell from edges of the cell.
 165. A methodaccording to claim 156, wherein the value of n in the multiplesubdivision operation varies dependent on the number of visual objectsin the cell.
 166. A method according to claim 164, wherein the value ofn in the multiple subdivision operation is calculated as directlyproportional to the quantity:

log_(q)(N/T)

where N is the number of objects in the cell, T is the subdivisionthreshold value, and q=s³.
 167. A method according to claim 162, 163 or164, wherein the data identifying the visual objects is stored so as tobe accessed by the coordinates of the cell and a level numberrepresenting the number of subdivisions by the subdivision numberbetween the cell and the first cell to which step (a) was applied in themethod.
 168. A method according to claim 167, wherein the dataidentifying visual objects is stored in a hash table structure accessedbased on the coordinates and level number of the cell.
 169. A methodaccording to claim 167, wherein the data identifying visual objectscomprises a stored pointer data value that points to another location inelectronic data storage at which the visual object data can be located.170. A method according to claim 167, wherein the data identifyingvisual objects is accessed by a content access memory using a key basedon the coordinates and level number of the cell.
 171. A method accordingto claim 156, wherein the visual object data of said visual objects isstored in a scene graph stored in the data storage device, and the dataidentifying the visual objects is a stored pointer value identifying atleast one location in the scene graph.
 172. A method according to claim156, wherein the visual object data of said visual objects is stored inRAM memory.
 173. A method according to claim 156, wherein one of thevisual objects stored is identified by a pointer to another database ofvisual objects prepared according to the present method couple with dataidentifying a transform matrix that spatially orients cells of thesecond database of visual objects relative to the coordinates of thecell.
 174. A method according to claim 156, wherein, when the operationof multiple subdivision is performed, a record is stored that representsthe integral power n of the multiple subdivision.
 175. A computer systemfor storing in a computer-readable data storage device a database ofscene data comprising electronic visual object data defining visualobjects each having associated therewith electronic data identifying alocation of the visual object in a virtual space, said systemcomprising: means for determining from the electronic visual object dataa number representing a count of the how many of the visual objects havelocations within a cell in the virtual space; means for comparing saidnumber of visual objects with a predetermined threshold subdivisionvalue and/or a predetermined threshold multiple subdivision value; meansfor performing, responsive to a determination that the number of visualobjects in the cell exceeds the predetermined threshold subdivisionvalue but not the predetermined threshold multiple subdivision value, asubdivision operation representing a partitioning of the cell in atleast one dimension thereof into an integer subdivision number s ofsub-cells that are equal in size to each other and together constitutethe cell; means for performing, responsive to a determination that thenumber of visual objects in the cell exceeds the predetermined thresholdmultiple subdivision value, a multiple subdivision operationrepresenting a partitioning of the cell in at least one dimensionthereof into a number of sub-cells, which number is an integral power,s^(n), of subdivision number s, said sub-cells being equal in size toeach other and together constituting the cell; means for storing dataidentifying the visual objects in the cell in the database so that thevisual object data of said visual objects can be accessed by a computerbased on coordinates identifying said cell, said means for storingstoring said data responsive to a determination that the number ofvisual objects in the cell does not exceed the predetermined thresholdsubdivision value, and means for providing to the means for determining,means for comparing, means for performing and means for storing each ofthe sub-cells created, said means for determining, means for comparing,means for performing and means for storing processing each sub-cell asthe cell for which the number of visual objects is counted.
 176. Asystem according to claim 175, wherein the operations representing apartitioning of the cell are of partitioning the cell in at least twodimensions thereof into s or s^(n) sub-cells that are equal in size toeach other and together constitute the cell.
 177. A system according toclaim 175, wherein the operations representing a partitioning of thecell are of partitioning the cell in three dimensions thereof into s³ ors^(3n) sub-cells that are equal in size to each other and togetherconstitute the cell.
 178. A system according to claim 175, wherein thesubdivision number is
 2. 179. A system according to claim 176, whereinthe subdivision number is
 2. 180. A system according to claim 177,wherein the subdivision number is 2, and where the subdivision operationyields eight sub-cells constituting the cell, and wherein the multiplesubdivision operation yields 2 sub-cells of the cell.
 181. A systemaccording to claim 175, wherein said cell is identified by integercoordinates thereof, and each sub-cell thereof is identified by at leastone coordinate derived by multiplying at least one of the coordinates ofthe cell by the subdivision number to obtain a product, and adding anon-negative integer number less than the subdivision number to theproduct as an indication of displacement from an edge of the cell. 182.A system according to claim 176, wherein said cell is identified byinteger coordinates thereof, and each sub-cell thereof is identified byat least two coordinates derived by multiplying at least two of thecoordinates of the cell by the subdivision number to obtain products,and adding a respective non-negative integer number less than thesubdivision number to each product as an indication of displacement ofthe sub-cell from edges of the cell.
 183. A system according to claim177, wherein said cell is identified by three integer coordinatesthereof, and each sub-cell thereof is identified by three coordinatesderived by multiplying the coordinates of the cell by the subdivisionnumber to obtain three products, and adding a respective non-negativeinteger number less than the subdivision number to each product as anindication of displacement of the sub-cell from edges of the cell. 184.A system according to claim 175, wherein the value of n in the multiplesubdivision operation varies dependent on the number of visual objectsin the cell.
 185. A system according to claim 177, wherein the value ofn in the multiple subdivision operation is calculated as directlyproportional to the quantity:

log_(q)(N/T)

where N is the number of objects in the cell, T is the subdivisionthreshold value, and q=s³.
 186. A system according to claim 181, 182 or183, wherein the means for storing stores the data identifying thevisual objects so as to be accessed based on the coordinates of the celland a level number representing the number of subdivisions by thesubdivision number between the cell and the first cell to whichsubdivision or multiple subdivision was applied by the system.
 187. Asystem according to claim 186, wherein means for storing includes datastorage comprising a hash table structure in which the data identifyingvisual objects is stored so as to be accessed based on the coordinatesand level number of the cell.
 188. A system according to claim 186,wherein the data identifying visual objects comprises a stored pointerdata value that points to another location in electronic data storage atwhich the visual object data can be located.
 189. A system according toclaim 186, wherein the means for storing stores the data identifyingvisual objects so as to be accessed by a content access memory using akey based on the coordinates and level number of the cell.
 190. A systemaccording to claim 175, wherein the means for storing stores the visualobject data of said visual objects in a scene graph stored in the datastorage device, and the data identifying the visual objects is a storedpointer value identifying at least one location in the scene graph. 191.A system according to claim 175, wherein the means for storing includesRAM memory storing the visual object data of said visual objects.
 192. Asystem according to claim 175, wherein one of the visual objects storedis identified by a pointer to another database of visual objects coupledwith data identifying a transform matrix that spatially orients cells ofthe second database of visual objects relative to the coordinates of thecell.
 193. A system according to claim 175, and further comprising meansfor storing for the multiply subdivided cell a record that representsthe integral power n of the multiple subdivision when the operation ofmultiple subdivision is performed.
 194. A memory for storing scene datafor access by a computer program being executed on a computer system,said scene data comprising data defining a plurality of visual objectseach having a respective virtual location in a scene space, said memorycomprising: a plurality of cell data entries stored in said memory, eachof said cell data entries corresponding to respective cell of the scenespace, said cell being defined by coordinates thereof and a subdivisionlevel value; the cell data entries being stored in said memory so thateach cell data entry is accessed based on the coordinates and levelvalue of the associated cell; said cell data entries each comprisingdata identifying the visual object or objects having a virtual locationwithin the cell; said cells being configured such that no more than athreshold subdivision number of visual objects have locations in anycell.
 195. A memory according to claim 194, wherein the cells aredefined as volumes of the scene space defined by integer boundaries of acoordinate grid, or as subdivisions derived therefrom, said subdivisionsbeing derived by portioning a larger cell in at least one spatialdimension thereof by a positive integer subdivision number s of equallyspaced partitions.
 196. A memory according to claim 195, wherein thesubdivisions are derived by partitioning in at least two dimensions.197. A memory according to claim 195, wherein the subdivisions arederived by partitioning in three dimensions.
 198. A memory according toclaim 195, wherein the subdivision number s is
 2. 199. A memoryaccording to claim 196, wherein the subdivision number s is
 2. 200. Amemory according to claim 197, wherein the subdivision number s is 2.201. A memory according to claim 195, wherein cells that have therein anumber of visual objects that exceeds a multiple subdivision thresholdnumber of visual objects are multiply subdivided by partitioning thecell in at least one dimension thereof into a number of sub-cells equalin size to each other, said number being an integral power, s^(n), ofsubdivision number s.
 202. A memory according to claim 201, wherein theinteger power n of multiple subdivision varies dependent upon the numberof visual objects in the cell.
 203. A memory according to claim 201,wherein the cell entry for a cell that is multiply subdivided includesdata recording the value of the integer power n of multiple subdivisionthereof.
 204. A memory according to claim 194, wherein the cells aredefined as volumes of the scene space defined by integer boundaries of acoordinate grid, or as subdivisions derived therefrom such that: cellsthat have therein a number of visual objects that exceeds a subdivisionthreshold number of visual objects, being subdivided by partitioning alarger cell in each of three spatial dimensions thereof by a positiveinteger subdivision number s, or an integral power (s^(n)) thereof, ofequally spaced partitions so as to yield s³ or s^(3n) subdivisionsthereof, and cells that have therein a number of visual objects thatexceeds a multiple subdivision threshold number of visual objects aremultiply subdivided by partitioning the cell in three spatial dimensionsthereof into a number of subdivisions equal in size to each other, saidnumber being an integral power of subdivision number s.
 205. A memoryaccording to claim 204, wherein the subdivision number s is
 2. 206. Amemory according to claim 204, wherein the coordinates of each cell are:a) coordinates of the associated grid position thereof; or b)coordinates derived from the coordinates of a cell that was subdividedto yield the cell.
 207. A memory according to claim 206, wherein thecoordinates are derived by multiplying the coordinates of the subdividedcell by the number of cells that the subdivided cell was subdivided ineach dimension.
 208. A memory according to claim 195, wherein the levelnumber of each cell represents the number of subdivisions into spartitions needed to reach the cell from the grid coordinate level,which is subdivision level zero.
 209. A computer system having a memoryaccording to one of claims 195 to 208, said computer system furthercomprising a tracing application program executable for identifying anyof the visual objects of the scene data that lie in a defined region ofscene space.
 210. A computer system according to claim 209, wherein saiddefined region of scene space is a path therethough.
 211. A computersystem according to claim 209, wherein said defined region of scenespace is a generally pyramidal beam extending therethough.
 212. Acomputer system according to claim 209, wherein said defined region ofscene space is a generally linear ray extending therethough.
 213. Amethod of rendering an image from scene data, said method comprising:providing to a computer system access to a memory according to one ofclaims 195 to 208; and accessing said memory so as to identify any ofthe visual objects of the scene data that lie in a defined region ofscene space.
 214. A method according to claim 213, wherein said definedregion of scene space is a path therethough.
 215. A method according toclaim 213, wherein said defined region of scene space is a generallypyramidal beam extending therethough.
 216. A method according to claim213, wherein said defined region of scene space is a generally linearray extending therethough.